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Abstract. The rock-paper-scissor game - which is characterized by three strategies R,P,S, sat- 
isfying the non-transitive relations S excludes P, P excludes R, and R excludes S - serves as a 
simple prototype for studying more complex non-transitive systems. For well-mixed systems where 
interactions result in fitness reductions of the losers exceeding fitness gains of the winners, classical 
theory predicts that two strategies go extinct. The effects of spatial heterogeneity and dispersal 
rates on this outcome are analyzed using a general framework for evolutionary games in patchy 
landscapes. The analysis reveals that coexistence is determined by the rates at which dominant 
strategies invade a landscape occupied by the subordinate strategy (e.g. rock invades a landscape 
occupied by scissors) and the rates at which subordinate strategies get excluded in a landscape 
occupied by the dominant strategy (e.g. scissor gets excluded in a landscape occupied by rock). 
These invasion and exclusion rates correspond to eigenvalues of the linearized dynamics near single 
strategy equilibria. Coexistence occurs when the product of the invasion rates exceeds the product 
of the exclusion rates. Provided there is sufficient spatial variation in payoffs, the analysis identi- 
fies a critical dispersal rate d* required for regional persistence. For dispersal rates below d*, the 
product of the invasion rates exceed the product of the exclusion rates and the rock-paper-scissor 
metacommunities persist regionally despite being extinction prone locally. For dispersal rates above 
d*, the product of the exclusion rates exceed the product of the invasion rates and the strategies 
are extinction prone. These results highlight the delicate interplay between spatial heterogeneity 
and dispersal in mediating long-term outcomes for evolutionary games. 

Author's Summary: The rock-paper-scissor game, which might initially seem to be of purely 
theoretical interest, plays an important role in describing the behavior of various real-world sys- 
tems including the evolution of alternative male mating strategies in the side-blotched lizard, the 
evolution of bacterial populations, and coexistence in plant communities. While the importance 
of dispersal in mediating coexistence for these intransitive communities has been documented in 
theoretical and empirical studies, these studies have, by in large, ignored the role of spatial het- 
erogeneity in mediating coexistence. We introduce and provide a detailed analysis of models for 
evolutionary games in a patchy environment. Our analysis reveals that spatial heterogeneity coupled 
with low dispersal rates can mediate regional coexistence, despite species being extinction prone 
in all patches. The results suggests that diversity is maintained by a delicate interplay between 
dispersal rates and spatial heterogeneity. 

Keywords: rock-scissors-paper game, non-transitive interactions, evolutionary game theory, 
replicator dynamics, spatial heterogeneity, metacommunity dynamics 
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Introduction 



Since its inception over 30 years ago evolutionary game theory has become a major theoretical 
framework for studying the evolution of frequency dependent systems in biology [Maynard Smith 



1982, Hofbauer and Sigmund 1998, 2003 . There have been numerous applications of evolutionary 



game theory in biology (and increasingly also in economics and the social sciences), ranging from 



the evolution of cooperation | Axelrod 1984, Axelrod and Hamilton, 1981] and animal conflicts 



Maynard Smith and Price, 1973 , to the evolution of sex ratios Hamilton, 1967 , and the origin of 



anisogamy Parker et al. , 1972| . Indeed it is striking that three of the simplest possible games that 
can be considered, the Prisoner's Dilemma game [Axelrod' 1984 , the Hawk-Dove (or Snowdrift) 
game [Maynard Smith 1982] , and the Rock-Paper-Scissor game [Hofbauer and Sigmund 1998| , 
have all found fruitful applications in the study of important biological problems, namely, the 
evolution of cooperation [Axelrod" 1984 Axelrod and Hamilton, 1981 , the evolution of animal 
contests 



Maynard Smith, 1982, Maynard Smith and Price, 1973], and the evolution of Red Queen 



dynamics |Sinervo and Lively 1996, Kerr et al. 2002 Kirkup and Riley, 2004 1 (in which the system 
cycles constantly between the different possible strategies). 

In formulating evolutionary game theory it is often assumed that the individual strategists 
interact at random in a well-mixed population. Under this assumption the evolutionary game 
dynamics can be formulated as a system of ordinary differential equations, the replicator equations, 
which describe the time evolution of the different strategies in the game |Maynard Smith] [1982 



Hofbauer and Sigmund , 1998 . Any evolutionary stable strategies (i.e. a strategy, which if adopted 



by almost all members of the population, cannot be invaded by any mutant strategy) is a stable 
equilibrium of the replicator equations [Hofbauer and Sigmund 1998 . 



In many situations the assumption that the population is well-mixed, with individuals interact- 
ing randomly throughout the whole population, is not realistic. This will often be the case if there is 
some spatial structure in the population, which results in individuals interacting more with neigh- 
boring individuals than with more distant ones. One way of modeling a structured population is to 
assume that individuals are associated with the vertices of a graph, with two individuals interacting 
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27 if the corresponding vertices are connected by an edge. This approach leads to a network based 

28 formulation of evolutionary game theory in which the evolutionary dynamics on the graph is deter- 

29 mined by a suitable deterministic or stochastic analogue of the replicator dynamics. Evolutionary 

30 games on graphs have been rather well studied [Nowak and May 1992, Killingback and Doebeli 



1996 Nakamaru et al. 1997, Hauert and Szabo 2003, Ifti et al. 2004 Hauert and Doebeli 2004 



Santos and Pacheco, 2005 Ohtsuki et al. 20061. One of the basic conclusions of this work is that 



33 the evolutionary dynamics of a game on a graph can be quite different from the dynamics of the 

34 game in a well-mixed population. A particularly important instance of this is that cooperation can 

35 be maintained in the Prisoner's Dilemma game on a graph. In contrast, in a well-mixed population 

36 cooperation is always driven to extinction by defection. 

37 An alternative way of modeling a structured population is to assume that it is composed of a 

38 number of local populations, within which individuals interact randomly, coupled by dispersal. In 

39 this approach the total population or community is modeled as a metapopulation or metacommu- 

40 nity. Metapopulation and metacommunity structure is known to have important implications for 



41 population dynamics in ecology and evolution Hanski, 1999 Holyoak et al. 2005 Schreiber, 2010] . 

42 In spite of the considerable amount of work that has been devoted to understanding the eco- 

43 logical and genetic consequences of metacommunity structure there has been much less attention 

44 devoted to studying the dynamics of evolutionary game theory in the metacommunity context. 

45 The purpose of this paper is to provide a general mathematical formulation of metacommunity 

46 evolutionary game dynamics, and to obtain detailed results for the case of a particularly interesting 

47 game - the rock-paper-scissors game. In the last few years the rock-paper-scissor game, which might 

48 initially seem to be of purely theoretical interest, has emerged as playing an important role in de- 

49 scribing the behavior of various real- world systems. These include the evolution of alternative male 



mating strategies in the side-blotched lizard Uta Stansburiana Sinervo and Lively, 1996 , the in 



51 vitro evolution of bacterial populations Kerr et al. , 2002, Nahum et al. , 2011 , the in vivo evolution 

52 of bacterial populations in mice |Kirkup and Riley , 2004 , and the competition between genotypes 



53 and species in plant communities Lankau and Strauss, 2007, Cameron et al. , 2009 . More generally, 
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54 the rock-scissors-paper game - which is characterized by three strategies R, P and S, which satisfy 

55 the non-transitive relations: P beats R (in the absence of S), S beats P (in the absence of R), and 

56 R beats S (in the absence of P) - serves as a simple prototype for studying the dynamics of more 



57 complicated non-transitive systems Buss and Jackson, 1979, Paquin and Adams, 1983 May and 



Leonard, 1975, Schreiber 1997 Schreiber and Rittenhouse 2004 Vandermeer and Pascual, 2005 



Allesina and Levine, 2011 



One of the central issues that has arisen in recent years in ecology is the degree to which meta- 



6i community structure can lead to the coexistence of competing species Hanski , 1999 , Amarasekare 



62 |and Nisbet 2001 Moquet et al. 2005, Gravel et al. 2010 . Here, we study an interesting aspect of 

63 this larger question, namely, the effect of a general metacommunity structure on the coexistence 

64 of the strategies in the rock-paper-scissor game. In a well-mixed population the evolutionary dy- 

65 namics of the rock-paper-scissor game is known to be determined by the sign of the determinant 

66 of the payoff matrix |Hofbauer and Sigmund 1998 . If the determinant of the payoff matrix is 

67 positive then the replicator dynamics converges to a stable limit point, in which the frequencies of 

68 the three strategies tend to constant values. If, however, the determinant of the payoff matrix is 

69 negative then the replicator dynamics converges to a heteroclinic cycle, in which the frequencies 

70 of the three strategies continue to undergo increasingly extreme oscillations. In the latter case the 

71 frequencies of the different strategies successively fall to lower and lower levels as the population 

72 dynamics approach the heteroclinic attractor. Consequently, stochasticity would result in the ul- 

73 timate extinction of one of the strategies followed by the elimination of the remaining dominated 

74 strategy. 

75 In this paper we study the dynamics of the rock-scissors-paper game in a metacommunity 

76 context, and show that dispersal in spatially heterogeneous environments can alter dynamical out- 

77 comes. In particular, we characterize under what conditions dispersal in heterogeneous environ- 

78 ments stabilizes or destabilizes rock-paper-scissor metacommunities. When dispersal is stabilizing, 

79 all strategies in the rock-scissors-paper metacommunity are maintained indefinitely by a Red Queen 
so type dynamic. 
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81 Model and Methods 

82 Evolutionary Games in Space. 

83 We consider interacting populations playing m distinct strategies (i = 1 , . . . , m) in a spatially 

84 heterogeneous environment consisting of n patches (r = 1, . . . ,n). Space is the primary limiting 

85 resource for the populations and assumed to be fully saturated i.e. all sites within a patch are 
se occupied. Let x\ denote the frequency of strategy i in patch r. Within patch reproductive rates of 

87 individuals are determined by pairwise interactions where an individual in patch r playing strategy 

88 i receives a payoff of Aij(r) following an encounter with an individual playing strategy j. Individuals 

89 reproduce at a rate equal to their net payoff. For individuals playing strategy i in patch r, this 

90 net payoff equals Aij(r)x^. All individuals in patch r experience a per-capita mortality rate rrf . 

91 Dying individuals free up space that can be colonized with equal likelihood by all offspring living 

92 in the patch. In the absence of dispersal, the probability that a site emptied by a dying individual 

93 gets colonized by an offspring playing strategy i is ^ - ? -, ■ Thus, in the absence of dispersal, 

94 the population dynamics in patch r are 

dx\ r r , r M r K*j m 

—L = - m r x r + m r , , r / r ■ (1) 

dt Y.j,k A jk{r)x r j x r k 

95 To account for movement between patches, let d sr denote the fraction of progeny born in patch s 

96 that move to patch r. In which case, the rate at which offspring of strategy i arrive in patch r equals 

97 J2 S dsr J2j Aij(s)xfx S j and the probability an offspring playing strategy i colonizes an emptied site 

98 equals ^ 'P V * J - j t~i s s • Hence, the full spatial dynamics are 

— f = -m r x[ + m r J s J s . 2 

99 We assume that the matrix D of dispersal probabilities is primitive (i.e. after sufficiently many 

100 generations, the decedents of any individual in any one patch occupy all patches). 

101 For the rock-paper-scissor game, there are three strategies with rock as strategy 1, paper as 

102 strategy 2, and scissor as strategy 3. Let a r be the basal reproductive rate of an individual in patch 

103 r. Let b\ (i.e. the benefit to the winner) be the payoff to strategy i in patch r when it wins against 
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104 its subordinate strategy, and — c\ (i.e. the cost to the loser) be the payoff to strategy i in patch r 

105 when it loses against the dominant strategy. Under these assumptions, the payoff matrix in patch 

106 r is given by 

/ -c\ b\ \ 
Mr) = a r +\ V 2 -c£ (3) 
\-c r 3 bl J 

107 Throughout this article, we assume that a r > 0, < c\ < a r , b\ > 0. The assumption a r > c\ 

108 ensures that payoffs remain positive. 

109 Analytical and Numerical Methods 

no To understand whether the strategies persist in the long-term, we analyze (|2j) using a combi- 

111 nation of analytical and numerical methods. Long-term persistence of all the strategies is equated 

112 with permanence: there exists a minimal frequency p > such that 

x\ (t) > p for all i, r 

in whenever t is sufficiently large and all strategies are initially present (i.e. Yl r x i(®) > for i = 
ii4 1,2,3). Permanence ensures that populations recover from rare large perturbations and continual 



ii5 small stochastic perturbations Schreiber, 2007, Bena'im et al. 2008 . Using analytical techniques 



ii6 developed by Hofbauer and Schreiber |2010 , we derive an analytical condition for permanence in 



ii7 terms of products of eigenvalues at the single strategy equilibria of the model. These criteria take 

us on an explicit, interpretable form when (i) populations are relatively sedentary (i.e. d rr ~ 1 for 

119 all r) and (ii) populations are well mixed (i.e. there exists a probability vector v = (vi, . . . ,v n ) 

120 such that d rs ~ v s for all r, s). To better understand permanence at intermediate dispersal rates, 

121 we derive an analytical result about critical dispersal thresholds for persistence of metacommunity 

122 exhibiting unconditional dispersal (i.e probability of leaving a patch is independent of location) and 

123 numerically simulate ^ using the deSolve package of R |Team 2008 . To simplify our exposition, 



124 we present our results under the assumption that rrf = m and a r = a for all r i.e. there is only 

125 spatial heterogeneity in the benefits and in the costs. More general results are presented in the 

126 Appendices. 
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(a) repelling cycle (b) attracting cycle 

Figure 1. Boundary dynamics for rock-paper-scissors. For within patch and metacommunity dynamics, there 
is a cycle of trajectories (i.e. heteroclinic cycle) connecting the pure strategy equilibria. In (a), the cycle is 
repelling and the community persists. In (b), the cycle is attracting and the community is extinction prone. 
Simulated metapopulations consist of 30 patches with all-to-all coupling for dispersing individuals and spatial 
variation in payoffs (c r = 1 + (r — 1) /30, b r = 0.85 c r , a = 3). The fraction dispersing equals d — 0.005 in (a) 
and d = 0.5 in (b). 



127 Results 

128 Local coexistence 

129 We begin by studying the behavior of the within-patch dynamics ([!]) in the absence of disper- 

130 sal. If only strategy i is present in patch r, then the per-capita growth rate of the strategy, call 

131 it j, dominated by strategy i is —mc^/a. Alternatively, the per-capita growth rate of the strat- 

132 egy j dominating strategy % equals mb\ja. The three single-strategy equilibria are connected by 

133 population trajectories in which dominant strategies replace subordinate strategies (Fig. [TJ. This 



134 cycle of population trajectories in patch j is known as a heteroclinic cycle Hofbauer and Sigmund 



135 ,1998 . Using average Lyapunov functions, time-one maps, or measure-theoretic techniques |Hof- 
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136 bauer, 1981, Krupa and Melbourne, 1995, Schreiber, 2000 , one can show that the strategies in 



137 patch r locally coexist in the sense of permanence provided that the product of the invasion rates 

138 exceeds the product of the exclusion rates: 



C; . 



(4) 



139 Interestingly, inequality Q is equivalent to the determinant of the payoff matrix being positive, 
uo When coexistence occurs, the heteroclinic cycle of the boundary of the population state space is 

ui repelling and there is a positive global attractor for the within-patch dynamics (Fig. [T^,) . When in- 

U2 equality Q is reversed, the heteroclinic cycle on the boundary is attracting (Fig.]!]}). The strategies 

143 asymptotically cycle between three states (rock-dominated, paper-dominated, scissor-dominated) , 

144 and the frequencies of the under-represented strategies asymptotically approach zero. Hence, all 

145 but one strategy goes extinct when accounting for finite population sizes. 



U6 Metacommunity coexistence. 

147 Analytical results. When the patches are coupled by dispersal, we show in Appendix A that 

us for any pair of strategies, the dominant strategy competitively excludes the subordinate strat- 

149 egy. Hence, as in the case of the dynamics within a single patch, the metacommunity exhibits a 

150 heteroclinic cycle on the boundary of the metacommunity phase space. 



i5i Work of Hofbauer and Schreiber [2010 on permanence for structured populations (see Appendix 



152 B) implies that metapopulation persistence is determined by invasion rates and exclusion rates at 

153 single strategy equilibria. More specifically, consider the rock strategy equilibrium where x\ = 1 



154 and x r n 



yields 



for all patches r. Linearizing the paper strategy dynamics at the rock equilibrium 



dx r 2 r . ^2 s d sr (a + b s 2 )x s 2 

— « -mx 2 + m — . 

at 2^ s d sr a 



156 Equivalently, if X2 



[x 2 , . . . ,x 2 



n\T 



where T denotes transpose, then 



dx 2 

~dt 



-ml + m^D T (aI + B 2 )) x 2 
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157 where I is the identity matrix, ^ is the diagonal matrix with entries 1/ ^ s d\ s a s , . . . , 1/ ^ s d ns a s , 

158 i?2 is the diagonal matrix with diagonal entries b\, ■ ■ ■ , , and -D T is the transpose of the dispersal 

159 matrix. Corresponding to the fact that the paper strategy can invade the rock strategy, the stability 

160 modulus of —ml + m^D T (aI + B2) (i.e. the largest real part of the eigenvalues) is positive. Call 

161 this stability modulus I2, the invasion rate of strategy 2. Linearizing the scissor strategy dynamics 

162 at the rock equilibrium yields 

~ « (-ml + m^D T (aI - C 3 )) x 3 

163 where C3 is the diagonal matrix with diagonal entries c§, . .. ,Cg. Corresponding to the fact that 

164 scissor strategy is displaced by the rock strategy, the stability modulus of —ml + m^D T (al — C3) 

165 is negative. We call this negative of this stability modulus S3, the exclusion rate of strategy 3. 

166 By linearizing around the other pure strategy equilibria, we can define the invasion rates X, for 

167 each strategy invading its subordinate strategy and the exclusion rates Si for each strategy being 

168 excluded by its dominant strategy. 

169 Appendix A shows that the metapopulation persists if the product of the invasion rates exceeds 

170 the product of the exclusion rates: 

3 3 

i=l i=l 

171 If the inequality ^ is reversed, then the metapopulation is extinction prone as initial conditions 

172 near the boundary converge to the heteroclinic cycle and all but one strategy is lost regionally. 

173 While inequality ^ can be easily evaluated numerically, one can not, in general, write down 

174 a more explicit expression for this permanence condition. However, when the metapopulation is 

175 weakly mixing (i.e. dispersal rates are low) or well-mixed (i.e. dispersal rates are high), we are able 

176 to find more explicit criteria. Furthermore, when dispersal is unconditional, we show that there is 

177 critical dispersal rate below which persistence is possible (Appendix C). 

178 At sufficiently low dispersal rates i.e d rr ~ for all r, the metacommunity coexistence crite- 

179 rion ([5]) simplifies to 

3 3 

jQmaxbJ" > JJminc[. (6) 



i=i i=i 
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180 Unlike the local coexistence criterion Q which requires that the geometric mean of benefits exceeds 

181 the geometric mean of costs within a patch, inequality ^ requires that the geometric mean of the 

182 maximal benefits exceeds the geometric mean of the minimal costs. Here, the maxima and minima 

183 are taken over space. Thus, inequality ^ implies that localized dispersal promotes coexistence if 

184 there is sufficient spatial variation in relative benefits, costs, or mortality rates. 

185 For well-mixed metacommunities (i.e. d rs ~ v s for all r,s), the invasion rate Xj of the strategy 

186 is approximately m J2 r K/ a - Conversely, the exclusion rate £j of strategy i is — m Yl r c i/ a - These 

187 well-mixed metacommunities coexist provided that the geometric mean of the spatially averaged 

188 benefit exceeds the geometric mean of the spatially averaged cost: 

n(s£«)>n(sE4 <r> 

i=l \ r / 1=1 \ r / 

189 Since ([7]) implies ([6]), it follows that persistence of well-mixed communities implies persistence of 

190 weakly-mixing communities, but not vice-versa. We can refine this observation under the assump- 

191 tion of unconditional dispersal. 

192 Unconditional dispersal occurs when the fraction of individuals dispersing, d, is independent of 

193 location. Let p rs denote the fraction of dispersing individuals from patch r that end up in patch s 

194 i.e. p sr is a dispersal kernel that describes how dispersing individuals redistribute across patches. 

195 Under these assumptions, the fraction d rs of individuals in patch r dispersing to patch s / r equals 

196 dp rs . The fraction d rr of individuals remaining in patch r is 1 — d. In Appendix C, we show that 

197 there is a critical dispersal threshold d* (possibly or 1) such that the metacommunity persists if 

198 its dispersal rate is below d* and is extinction prone when its dispersal rate is greater than d* . It 

199 follows that if the metacommunity persists when highly dispersive (i.e. d* = 1), then it persists at 

200 all positive dispersal rates. Conversely, if a metacommunity is extinction prone when weakly mixing 

201 (i.e. ^ is violated), then it is extinction prone at all positive dispersal rates. 

202 Numerical results. To illustrate the implications our analytical results, we consider two scenarios 

203 where either there is only spatial variation in the payoffs or where there is within-patch and spatial 

204 variation in payoffs. There are n = 30 patches that are equally connected. A fraction d of individuals 

205 disperse and dispersing individuals are distributed equally amongst the remaining patches (i.e. 
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(b) 



Figure 2. The effect of spatial variation and dispersal rate on the persistence criterion in (a) and the long-term 
metapopulation frequencies in (b) . Metapopulations consist of 30 patches with all-to-all coupling for dispers- 
ing individuals and spatial variation in payoffs (c r = l + (r— 1)<t/30, b r = 0.85 c r , a — 3). In (a), the difference 
between the product JT- 01 the invasion rates and the product \\ i Ei of the exclusion rates are plotted 
as function of the fraction d of dispersing individuals and the range a of spatial variation in the payoffs. 
Positive values correspond to persistence and negative values to the metapopulation being extinction prone. 
The white curve is where the difference in products equals zero. In (b), the minimal and maximal frequencies 
for one patch and the spatial average are plotted as a function of the fraction d of dispersing individuals and 
cr. The white curve is where the difference in the products of invasion and exclusion rates equals zero. 



206 d rs = d/(n— 1) for r ^ s). For this form of dispersal, the metapopulation is well-mixed when 

207 d = (n — l)/n in which case d rs = 1/n for all r, s. 

208 First, we consider the case where there is spatial variation in payoffs, but all strategies within a 

209 patch fare equally well when they are the dominant player in an interaction and fare equally poorly 

210 when they are the subordinate player in the interaction (i.e. b\ = b r , and c\ = c r for all %= 1, 2, 3). 
2n Local coexistence requires that the benefit b r to the winner must exceed the cost c r to the loser. For 

212 well-mixed communities, regional coexistence requires that the spatially averaged benefit ^ ^ r b r 

213 must exceed the spatially averaged average cost ^ ^ r c r . From these two conditions, it follows that 

214 metapopulation persistence for well-mixed communities requires that at least one of the patches 

215 promotes local coexistence. 
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Figure 3. The effect of dispersal rates on metapopulation dynamics. Metapopulations consist of 30 patches 
with all-to-all coupling for dispersing individuals and spatial variation in payoffs (c r = 1 + (r — l)/30, 
b r = 0.85 c r , a = 3). In (a), the minimal and maximal frequencies for one patch and the spatial average are 
plotted as a function of the fraction d of dispersing individuals. In (b)-(d), the spatial-temporal dynamics 
are plotted for low, intermediate, and high dispersal rates. Rock frequencies are color-coded as indicated. 



216 When all patches fail to promote local coexistence (i.e. c r > b r for all r), weakly mixing meta- 

217 communities persist provided that the benefit in some patch exceeds the cost in another (possibly 

218 the same) patch i.e. max r b r > min r c r . When this condition is meet, there is a critical dispersal 

219 threshold d* below which the metacommunity persists, and above which the metacommunity is 

220 extinction-prone. 

221 Figure [2^i demonstrates the analytical prediction that the difference between the products of 

222 the invasion and exclusion rates is a decreasing function of the fraction d dispersing. Furthermore, 

223 the difference in products is an increasing function of the amplitude of the spatial variation in 

224 payoffs. Hence, the critical dispersal threshold increases with the amplitude of the spatial variation 

225 of the payoffs. Intuitively, higher dispersal rates are needed to average out greater spatial variation. 

226 Unlike the difference between the products of invasion and exclusion rates, the minimum frequency 

227 of strategies exhibits a highly nonlinear response to increasing dispersal rates (Fig[2b): the minimal 
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(a) (b) 

Figure 4. The effect of spatial variation and dispersal rate on the persistence criterion in (a) and long-term 
metapopulation frequencies in (b). Mctapopulations consist of 30 patches with all-to-all coupling for dis- 
persing individuals. Each strategy has 10 patches in which their benefit equals bhigh and equals in the 
remaining patches, c = 1, a = 2, m = 0.1 in all patches. In (a), the difference between the product Yii-^i °f 
the invasion rates and the product Y[i &i of the exclusion rates are plotted as function of the fraction d of 
dispersing individuals and the maximal benefit bhigh- Positive values correspond to persistence and negative 
values to the metapopulation being extinction prone. The white curve is where the difference of products 
equals zero. In (b), the minimal and maximal frequencies for one patch and the spatial average are plotted 
as a function of the fraction d of dispersing individuals and the maximal benefit bhigh- The white curve is 
where the difference in the products of invasion and exclusion rates equals zero. 



228 frequency initially increases with dispersal rates, reaches a plateau of approximately one-third at 

229 intermediate dispersal rates, and decreasing abruptly to zero after crossing the critical dispersal. 

230 At low dispersal rates, metacommunity persistence is achieved by a spatial game of hide and 

231 seek (Figs. [3^,b). At any point in time, each strategy is at high frequency in some patches and 

232 low frequencies in the remaining patches. Strategy composition in each patch cycles as dominant 

233 strategies displace subordinate strategies. Intermediate dispersal rates stabilize the local and re- 

234 gional dynamics (Figs. [3^,, c). As a consequence, local diversity is maximal at intermediate dispersal 

235 rates. At high dispersal rates, the population dynamics synchronize across space as they approach 

236 the heteroclinic cycle (Figs. [3^i,d). 

237 For the second numerical scenario, we consider when payoffs vary within patches (e.g. rock gets 
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fraction dispersing 



Figure 5. The effect of dispersal rates on metapopulation dynamics. Metapopulations consist of 30 patches 
with all-to-all coupling for dispersing individuals. Each strategy has 10 patches in which their benefit equals 
bhigh = 4 and equals in the remaining patches, c = 1, a = 2, m = 0.1 in all patches. In (a), the minimal 
and maximal frequencies for one patch and the spatial average are plotted as a function of the fraction d of 
dispersing individuals. In (b)-(d), the spatial-temporal dynamics are plotted for low, intermediate, and high 
dispersal rates. Rock frequencies are color-coded as indicated. 



238 a higher benefit than scissor when playing their subordinate opponents in one patch, but scissor 

239 gets the higher benefit in another patch) as well as spatially. In this case, well-mixed communities 

240 can persist despite being locally extinction prone. To understand why, assume each strategy wins 

241 big in some patches but win nothing in others. Let / denote the fraction of patches where a 

242 strategy wins big and receives a payoff bhigh against its subordinate strategy. In the remaining 

243 fraction 1 — / of patches, each strategy receives no benefit when playing against their subordinate 

244 strategy. Furthermore, assume that there is no variation in the costs c[ = c for all i,r. Under 

245 these assumptions, local coexistence is impossible as c > bhigh • = 0. In contrast, a well-mixed 

246 metacommunity persists if / bhigh > c and a weakly-mixing metacommunity persists if bhigh > c - 

247 Therefore, provided that bhigh is sufficiently large, coupling the communities by any level of dispersal 

248 mediates regional coexistence despite local communities being extinction prone. 

249 Consistent with these analytical predictions, Fig. [4] illustrates that metacommunity persists at 

250 all dispersal rates if the difference in payoffs is sufficiently great (bhigh > 3) and only persists at 
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251 low dispersal rates for intermediate differences in the payoffs. When there are large differences, 

252 metapopulation abundance and stability increases continually with dispersal rates (Fig. [5]). In 

253 contrast, metapopulation abundance is maximized at intermediate dispersal rates whenever there 

254 are intermediate differences in the payoffs (Fig. Hp) . 



Discussion 



256 The rock-paper-scissors game represents the prototypical situation in which the components of a 

257 system satisfy a set of non-transitive relations. It is a surprising and fascinating feature of recent 

258 work in evolutionary biology and ecology that such interactions have been discovered in a wide 



259 range of natural systems Buss and Jackson, 1979, Sinervo and Lively, 1996, Kerr et al. 2002 



Kirkup and Riley, 2004, Lankau and Strauss, 2007, Cameron et al. 2009 . The existence of non 



261 transitive interactions in biological systems has been suggested as an important mechanism for 

262 maintaining biodiversity |Durrett and Levin 1997, Kerr et al. 2002, Lankau and Strauss, 2007 



Roelke and Eldridge, 2010, Allesina and Levine, 2011 . This suggestion, however, raises an important 



264 theoretical question: Is it possible for all components of such a system to persist in the long term? 

265 This question is pertinent since modeling the dynamics of the rock-paper-scissors game (and related 

266 non-transitive systems) using the replicator equation shows that cyclic behavior corresponds to 

267 convergence toward a heteroclinic attractor on the boundary of the strategy space, and this process 



268 must ultimately result in the extinction of some strategies Hofbauer and Sigmund, 1998 



269 It is widely believed in ecology that the inclusion of spatial structure, in which the interactions 

270 of individuals are local, can result in the coexistence of communities that could not persist in a 



27i ^anmictic situation [Durrett and Levin , 1997 Hanski 1999 , Amarasekare and Nisbet 2001 , Holyoak 



272 et al. 2005 1 . There are numerous ways in which a spatially structured population can be modeled 



273 mathematically, depending on the assumptions made regarding the nature of the spatial interactions 



274 of the individuals in the population |Durrett and Levin, 1994 . Possible approaches include reaction 



275 diffusion systems Cantrell and Cosner, 2003 , metapopulation and metacommunity theory Hanski 



276 1999 , Holyoak et al. 2005 , coupled lattice maps Hastings , 1993 , Holland and Hastings , 2008 , and 
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277 cellular automata and related lattice models |Nowak and May 1992 Killingback and Doebeli 1996 



1997 


1998, 


Iwasa et al. 


1998, 



279 Most previous attempts to understand the effect of spatial structure on the persistence of sys- 



280 |£ems with non-transitive interactions have utilized cellular automata-type models |Durrett and 

281 |Levin 



1997, 


1998, 


Iwasa et al. 


1998 



2005, Reichenbach et al. 2007, Rojas-Echenique and Allesina, 2011 . The main conclusion that can 



283 be drawn from these cellular automata studies is that in three-species systems with non-transitive 

284 interactions it is possible for all species to coexist in a spatially structured model even when they 

285 could not all persist in the corresponding panmictic system. Coexistence in these models when 

286 formulated in two spatial dimensions results from the different species aggregating in regions that 



287 ipyclically invade each other. It is worth noting that in the reaction-diffusion approach of Nakamaru 

288 |and Iwasa [2000] coexistence is not possible in one-dimensional systems. This issue has not, how- 

289 ever, been investigated using lattice models. Cellular automata models have the virtue of explicitly 

290 introducing space through a lattice of cells and of directly modeling the spatial interactions be- 

291 tween individuals. However, such models also have a number of significant limitations. Since spatial 

292 structure is introduced in a very concrete fashion, through an explicit choice of a spatial lattice 

293 (almost always taken to be a two-dimensional square lattice) and a spatial interaction neighborhood 

294 (usually taken to be the eight cells surrounding the focal cell) it is, in general, unclear how changes 

295 in these structures affect species coexistence. A second limitation of cellular automata models is 

296 the difficulty is using them to study the effects of spatial heterogeneity. In all the lattice models 

297 of non-transitive interactions that have been studied the rules determining how cells are updated 

298 are the same at every spatial location, although it is known, in general, that spatial heterogeneity 

299 may have important implications for species coexistence |Amarasekare and Nisbet 2001 . A third 

300 limitation is that cellular automata are notoriously difficult to study analytically, and indeed almost 

301 all the key results on coexistence of species with non-transitive interactions in lattice models have 

302 been obtained from simulations (see, however, Durrett 2009Q. 

303 In this paper we have adopted the metacommunity perspective to formulate a new approach 
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304 to studying the dynamics of spatially structured communities in which rock-paper-scissors-type in- 

305 teractions hold. This approach assumes that the overall metacommunity is composed of a number 

306 of local communities, within each of which the interactions are panmictic, and that the local pop- 

307 ulations are coupled by dispersal. The resulting metacommunity model allows for a very general 

308 treatment of the population dynamics of spatially structured systems with non-transitive inter- 

309 actions, which overcomes many of the limitations inherent in cellular automata-type models. In 

310 particular, our model allows a very general treatment of dispersal between spatial patches, includes 

311 spatial heterogeneity in a fundamental way, and allows precise analytic derivations of the central 

312 results. 

313 In our model, in the absence of dispersal, the population dynamics within each patch exhibits 

314 a heteroclinic cycle. Coexistence of all strategies in any given patch requires that the geometric 

315 mean of the benefits obtained from the payoff exceed the geometric mean of the costs within that 

316 patch. Moreover, when the spatial patches are coupled by dispersal the metacommunity possesses 

317 a heteroclinic cycle, and all members of the metacommunity persist when a regional coexistence 

318 criterion holds-the geometric mean of invasion rates when rare of the dominant strategies exceed 

319 the geometric mean of the exclusion rates when rare of the subordinate strategies. Although it is 

320 not possible, in general, to write down an explicit formula for the eigenvalues associated with these 

321 invasion and exclusion rates, it is possible to find more explicit expressions in the limiting cases of 

322 weakly-mixed metacommunities and well-mixed metacommunities. Weak mixing occurs when when 

323 dispersal rates are low. In this case, our analysis reveals that sufficient spatial heterogeneity in the 

324 payoffs for pairwise interactions allows metacommunity coexistence even when every local commu- 

325 nity is extinction prone. Thus, in the presence of spatial heterogeneity, local dispersal promotes 

326 coexistence. Alternatively, when dispersal rates are high, the metacommunity is well-mixed. In this 

327 case, the coexistence criterion requires that the geometric mean of spatially averaged benefits ob- 

328 tained from the payoff exceed the geometric mean of the spatially averaged costs. These coexistence 

329 criteria imply that the coexistence of a well-mixed metacommunity guarantees the coexistence of 
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330 the corresponding weakly mixed one. The converse result does not hold. Thus, metacommunities 

331 with higher dispersal rates are less likely to persist than those with lower ones. 

332 For unconditional dispersal (i.e. when the fraction d of individuals dispersing is independent 

333 of location), the interaction between spatial heterogeneity and dispersal leads to a threshold ef- 

334 feet: there exists a critical dispersal value d* , such that if the dispersal rate is less than d* the 

335 metacommunity persists, while if the dispersal rate is greater than d* it is extinction prone. This 

336 threshold effect occurs whenever well-mixed communities are extinction prone but weakly-mixed 

337 communities are not. For example, there is sufficient spatial variation in the payoffs but the cost 

338 paid by the loser exceeds the benefit gained by the winner in every pairwise interaction. Similar 

339 dispersal thresholds have been demonstrated for two-species competitive communities exhibiting 



340 either priority effects or local competitive dominance ILevin 1974 Amarasekare and Nisbet 20011. 



341 However, unlike these transitive systems, regional coexistence for these intransitive systems does 

342 not require each species having regions in space where either they are initially more abundant or 

343 competitively dominant. 

344 Our results on the effect of dispersal on the coexistence of rock-paper-scissors metacommunities 

345 are in broad qualitative agreement with the conclusions that can be drawn from cellular automata- 

346 type models that include the movement of individuals, which is the lattice analogue of dispersal. 

347 Karolyi et al. |2005 considered a two-dimensional lattice model of non-transitive interactions in 



348 n which individuals moved due to a chaotic flow, such as might occur in a fluid system. Reichenbach 



et al. 2007 also studied the effect of mobility on coexistence in a two-dimensional cellular automata 



350 model of rock-paper-scissors interactions, where individual movement was modeled using techniques 



35i of dimmer automata Schofisch and Hadeler, 1996 . In each case it was found through simulation 



352 that there exists a critical level of mobility, below which all species coexist and above which only 

353 one species survives in the long-term. This critical mobility level in lattice models of rock-paper- 

354 scissors interactions is the analogue of the critical dispersal rate d* in our metacommunity model. 

355 It is interesting to note in this context that a similar threshold also occurs in a model of cyclic 



356 interactions on complex networks studied by Szabo et al. 2004 . In this case if the fraction of 
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357 long-range interactions present in a small-world network is below a critical value coexistence of all 

358 species is possible, while if it is exceeded species extinctions occur. 

359 We also note that a further example of a lattice model that has been used to study the effect 

360 of spatial structure in maintaining meta-community persistence in a system with non-transitive 



361 interactions occurs in the area of prebiotic evolution. Eigen and Schuster 1979 observed that there 

362 is a fundamental problem in the evolution of self-replicating molecules: there exists an information 

363 threshold since the length of the molecule is restricted by the accuracy of the replication process. 

364 Eigen and Schuster proposed as a solution to this problem the concept of the hypercycle, in which 

365 a number of molecules catalyze the replication of each other in a cyclic fashion. The dynamics 

366 of a hypercycle can be modeled mathematically as a replicator equation with a cyclic payoff ma- 



367 trix |Hofbauer and Sigmund, 1998 J, and thus the hypercycle corresponds dynamically to a replicator 



368 system with non-transitive interactions. The concept of a hypercycle has, however, a crucial flaw: 

369 it is not evolutionarily stable - selection will favor the evolution of a parasitic mutant which does 

370 not provide any catalytic support to other molecules in the hypercycle even though it receives such 

371 catalytic support itself |Maynard-Smith 1979, Bresch et al. 1980 . The evolution of parasitic mu- 



372 tants results in the collapse of hypercycles as entities capable of encoding information. Interestingly, 

373 the inclusion of spatial structure can prevent the evolution of selfish mutants and may result in 

374 the persistence of hypercycles. The effect of spatial structure on the persistence of hypercycles has 

375 been studied using a cellular automaton model in |Boerlijst and Hogeweg 1991 . It is shown in 



376 this model that local spatial interactions result in the formation of self-organized spiral waves, and 

377 that selection acting between these spiral waves can counteract the effect of selection acting at the 

378 level of the individual molecules, with the consequence that the hypercycle can be resistant to the 

379 evolution of parasitic mutants. 

380 The metacommunity model we have introduced here provides a complementary approach to 

381 the lattice models that have previously been used to study coexistence in rock-paper-scissors-type 

382 systems. It seems likely that each type of model will most naturally describe different types of 

383 empirical systems with non-transitive interactions. For example, the lattice modeling approach 
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384 may describe reasonably well an in vitro microbial population growing on a plate Kerr et al 



385 2002 . In contrast, our metacommunity model would seem to be a more natural approach to use 

386 to describe an in vivo microbial population inhabiting many host organisms with transmission 

387 between the hosts, as in the model system of Kirkup and Riley [2004], or plant communities living 



388 on different soil types Lankau and Strauss, 2007, Cameron et al. 2009| . This observation raises 



389 the possibility that it may be possible to use such systems to empirically test the predictions of our 

390 metacommunity model. 
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